A Note on Sequences of Continuous Parameter Markov Chains

Continuous Time Markov Chains

Discrete-time Markov chains are useful in simulation, since updating algorithms are easier to construct in discrete steps. They can also be useful as crude models of physical, biological, and social processes. However, in the physical and biological worlds time runs continuously, and so discrete-time mathematical models are not always appropriate. This is especially true in population biology –...

Continuous-time Markov Chains

A note on faithful coupling of Markov chains

One often needs to turn a coupling (Xi, Yi)i≥0 of a Markov chain into a sticky coupling where once XT = YT at some T , then from then on, at each subsequent time step T ′ ≥ T , we shall have XT ′ = YT ′ . However, not all of what are considered couplings in literature, even Markovian couplings, can be turned into sticky couplings, as proved by Rosenthal [Ro97] through a counter example. Rosenth...

On Markov Chains with Continuous State Space

In this expository paper, we p r o ve the following theorem, which m a y be of some use in studying Markov chain Monte Carlo methods like hit and run, the Metropolis algorithm, or the Gibbs sampler. Suppose a discrete-time Markov c hain is aperiodic, irreducible, and there is a stationary probability distribution. Then from almost all starting points the distribution of the chain at time n conv...

On Line Identification of the First Markov Parameter of Linear Multivariable Plants (RESEARCH NOTE)

In this paper three methods for on-line identification of first markov parameter at linear multivariable plants are presented. In these methods input-output data are used far the on-line identification of the first markov parameter.